Coaching Advice and Adaptation

As agent relationships become more complex, one challenging relationship that merits study is that of coach or adviser to another agent. Our research on coaching refers to one autonomous agent providing advice to another autonomous agent about how to act. In this paper, we first describe the current state of the coach agent framework we are developing. One of our goals is to design a coach agent that can provide advice to many differently built and structured advice taking agents. A coach must then adapt its advice to the capabilities and limitations of the agents it is coaching. This paper contributes our principled statement and exploration of the effect of advice giving in the presence of limitations. We develop and empirically test our hypotheses in a predator-prey grid world, which abstracts from the complexities of other agent domains we have used in the past and allowed for this precise study. The predator is a Q-learning agent that can take action advice from a coach agent. The coach begins with optimal policies for the predator. During the coach-agent interaction, the coach provides advice to the predator with the goal of helping to improve its performance. We experimentally explore a variety of situations: (i) limiting the actions that the predator can perform, (ii) limiting the amount and frequency of advice the coach provides, (iii) limiting the amount of memory the predator agent has for advice, and (iv) changing whether the coach can see the predator’s actions. The results show that coaching can improve agent performance in the face of all these limitations.